Structured classification for multilingual natural language processing

This thesis investigates the application of structured sequence classification models to multilingual natural language processing (NLP). Many tasks tackled by NLP can be framed as classification, where we seek to assign a label to a particular piece of text, be it a word, sentence or document. Yet often the labels which we’d like to assign exhibit complex internal structure, such as labelling a sentence with its parse tree, and there may be an exponential number of them to choose from. Structured classification seeks to exploit the structure of the labels in order to allow both generalisation across labels which differ by only a small amount, and tractable searches over all possible labels. In this thesis we focus on the application of conditional random field (CRF) models (Lafferty et al., 2001). These models assign an undirected graphical structure to the labels of the classification task and leverage dynamic programming algorithms to efficiently identify the optimal label for a given input. We develop a range of models for two multilingual NLP applications: word-alignment for statistical machine translation (SMT), and multilingual supertagging for highly lexicalised grammars. The first half of this thesis is dedicated to the task of word-alignment for SMT, which aims to find a mapping from words in a source language sentence to words in a target translation sentence. We treat this problem as one of structured classification with a CRF, where the input is the parallel sentence pair and the output is the index for each word in the source sentence of its aligned translation (or null) in the target sentence. By exploiting the ability of the CRF model to incorporate a diverse range of features we are able to explore many binary and real-valued features. Orthographic and syntactic features are defined which aim to generalise sparse word-to-word translation features. In addition, we define powerful features from unsupervised generative models, and collocation statistics derived